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Abstract 
In the U.S., many states have adopted Response to Intervention or Multi-tiered Systems of 
Supports to provide early intervention. However, there is considerable variability in how states 
and schools implement RTI. Teachers are responsible for using student data from RTI to inform 
instructional decisions for students with or at risk for dyslexia, so it is necessary to understand 
the knowledge they have about the structure of RTI in their individual schools. This study 
reviews the results of an exploratory factor analysis of a survey aimed at measuring teachers’ 
knowledge about RTI implementation and their understanding of RTI implementation within 
their school. The 52-item survey was administered online to 139 general and special education 
teachers. The three final factors from this factor analytic work were: (1) Teacher Knowledge 
about Tier 1 Implementation; (2) Teacher Knowledge about Leadership and School Systems, and 
(3) Teacher Knowledge about Data-Based Decision Making. Factor determinacy scores 
demonstrated that the survey had high internal consistency. On average, teachers’ survey scores 
were higher on the first two factors, and slightly lower on the third factor. Implications of the 
findings for teachers of students with learning disabilities, including dyslexia, and directions for 
future research were discussed. 
Keywords: Factor analysis, Multi-tiered Systems of Support, Response to Intervention, Teacher 


knowledge survey 
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Elementary Teacher’s Knowledge of Response to Intervention Implementation: 
A Preliminary Factor Analysis 

In the U.S., the reauthorization of the Individuals with Disabilities Education Act (IDEA; 
2004) added language specifying that states could adopt Response to Intervention (RTI). This 
new language specified that the purpose of RTI was to identify struggling students early, to 
provide them with evidence-based interventions, to closely monitor their progress, and to adapt 
or intensify interventions based on progress monitoring data (Fuchs, Fuchs, & Stecker, 2010). 
Providing intensive early intervention was intended to prevent many reading difficulties. Another 
intention was that RTI could replace the IQ achievement discrepancy approach to identifying 
students with reading disabilities such as dyslexia; that approach had become known as a wait- 


to-fail approach (Fuchs et al., 2010). 


RTI is currently in use in every state within the U.S. (Zirkel & Thomas, 2010). The 
Institute for Education Sciences reviewed the existing research base for RTI and issued a practice 
guide (Gersten et al., 2008) detailing recommendations for implementation to support struggling 
readers, including students with dyslexia. These evidence-based recommendations include 
universal screening of all students, provision of explicit and systematic Tier I reading instruction, 
and additional tiers of increasing intensity for students demonstrating inadequate response to 
instruction. Tier 2 supports should include intensive and systematic instruction provided 
regularly and in small groups and frequent progress monitoring. Tier 3 should include more 
intensive and individualized instruction for students who do not respond to Tier 2 supports. 

However, these evidence-based recommendations have not been consistently 
incorporated into procedural guidance for RTI implementation, which varies from state to state, 


and often varies among districts in the same state (Jenkins, Schiller, Blackorby, Kalb Thayer, & 
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Tilly, 2013; Mellard, McKnight, & Jordan, 2010; Mellard, McKnight, & Woods, 2009). Further- 
more, recent legislation has introduced more variability; the Every Student Succeeds Act (ESSA, 
2015) introduced the term “‘multi-tiered systems of support (MTSS)” which represents a broader 
constellation of supports than RTI. MTSS incorporates not only academic outcomes but also 
behavior and social emotional learning. ESSA provided little additional procedural guidance or 
clarification for RTI or MTSS implementation, and emphasized states and local education 
agencies may develop their own models. 

Variable RTI implementation may be related to the mixed findings about the impact of 
RTI when implemented within typical school practice, despite converging findings of positive 
effects from researcher-implemented RTI. For example, positive findings were reported in one 
large-scale study involving 318 elementary schools in the state of Florida (Torgesen, 2007). The 
Florida RTI model followed evidence-based practices, and it included explicit and systematic tier 
1 reading instruction, universal reading screening, evidence-based interventions, and ongoing 
progress monitoring. Torgesen reported significantly fewer students were identified with reading 
disabilities after schools began implementing RTI. By contrast, less encouraging findings about 
RTI implementation were reported in another large-scale study examining data from 146 schools 
across the USA (Balu et al., 2015). The researchers collected data from 20,450 first- through 
third-grade students and used a regression discontinuity design, which examined whether 
students who qualified for tier 2 or tier 3 intervention (by testing just below a cut score on a 
benchmark screener) scored higher than students performing just above the cut score (and 
therefore did not qualify for interventions). Findings that indicated second- and third-grade 
students who qualified for tier 2 and tier 3 interventions showed no statistically different 


outcomes than students who did not. At first grade, students who qualified for interventions 
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performed statistically worse than their peers. However, researchers have cau- tioned against 
interpreting Balu et al.’s (2015) findings to mean that RTI does not work, because many schools 
did not consistently implement RTI using evidence-based practices (e.g., Fuchs & Fuchs, 2017; 
Gersten, Jayanthi, & Dimino, 2017). 

The success of RTI is largely dependent on elementary teachers’ knowledge about RTI 
implementation, because these teachers are the first line of defense against reading difficulties. 
For students who are struggling, or who have reading disabilities, including dyslexia, teachers 
need to know (1) how to identify who needs help, (2) what help to provide, and (3) how to access 
the resources within their school and district. A primary aim of our research study is to describe 
preliminary findings from a survey we conducted of elementary teachers’ knowledge about RTI 
implementation within their own school settings and an analysis of the factors derived from the 
results. First, we describe the conceptual and empirical framework for our study, based on the 
existing research that has surveyed teacher knowledge of RTI and, given the limitations of this 
research base, we also describe surveys of teacher knowledge about reading instruction. Then, 
we highlight the limited psychometric work that has been conducted to date. 

Surveying Teachers’ Knowledge about RTI 

In order for RTI to be successful for students with reading disabilities and dyslexia, 
teachers need knowledge about how to use data to identify students’ level of performance 
relative to either peers or benchmark assessments and how to develop instructional plans related 
to their relative strengths and weaknesses. To date, a fairly limited number of studies have 
surveyed various aspects of teachers’ knowledge about RTI (i.e., how to identify who needs help, 
what interventions are needed, and what resources are within the school). However, findings 


across previous research have been very consistent. Studies have revealed that teachers often 
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report that they understand broadly what RTI is, how to administer assessments, and how to 
locate data (Barnes & Burchard, 2011; Spear-Swerling & Cheesman, 2012; Wilcox, Murakami- 
Ramalho, & Urick, 2013). However, teachers also report having little knowledge of what to do 
with that information to make instructional decisions to help their students, particularly those 
who have or are at risk for developing reading disabilities (Means, Padilla, DeBarger, & Bakia, 
2009; Spear-Swerling & Cheesman, 2012; Vujnovic et al., 2014; Wilcox et al., 2013). For 
example, the report of the National Study of Education Data Systems and Decision Making 
(Means et al., 2009), which surveyed 2509 teachers about their use of and ability to access data, 
reported that 81% of teachers knew where to access student data, but fewer than half could 
interpret the data to change instruction. A few years later, Spear-Swerling and Cheesman (2012) 
developed a 66-item Teacher Knowledge Survey for RTI to assess teacher familiarity with and 
knowledge of foundational reading skills, identification and use of assessments, and knowledge 
about RTI. A total of 142 preservice and in-service teachers participated: the authors reported an 
adequate Cronbach’s alpha of 0.88. Findings revealed 85% of the teachers had basic knowledge 
of RTI, but fewer than one third reported having the knowledge to use the data from assessments 
to adjust instruction. Moreover, a majority could not identify tier 2 and tier 3 reading programs, 
and they had little knowledge generally about evidence-based reading practices such as findings 
from the National Reading Panel (2000). Similarly, Vujnovic et al. (2014) developed and tested 
the 30-item Ready for Response to Intervention Questionnaire which they administered to 308 
general educators, special educators, and school psychologists. The items addressed 
understanding of and ability to apply principles of RTI. Reliability data was reported only for 
questions related to how teachers rated their knowledge but not for how they applied that 


knowledge. Internal consistency for that section was reported as high with an alpha of 0.97. The 
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researchers found that less than half of the respondents could identify which tier of instruction 
would be most beneficial for students given a range of scenarios of students with varied reading 
abilities. 

Findings from a qualitative study described teachers’ mixed perceptions about their self- 
efficacy for implementing RTI (Wilcox et al., 2013). The researchers conducted focus groups, 
interviews, and questionnaires with 81 elementary teachers across two states. No reliability data 
were available for the instruments. Findings suggested that teachers consistently believed 
themselves to be capable of assessing and identifying students in need of support; however, 
teachers reported having less confidence about their knowledge about providing intensive 
interventions for these students. 

To summarize, within this emerging database, converging findings support teachers’ need 
for additional support and professional development. It is important to interpret these findings in 
light of the variability in RTI implementation and the lack of procedural guidance about RTI. 
Further, some researchers provided support that their survey instruments had adequate reliability 
(Spear-Swerling & Cheesman, 2012; Vujnovic et al., 2014), while others did not provide 
reliability information (Wilcox et al., 2013). 

Surveying Knowledge about Teaching Reading 

A relatively larger research base involves surveys to describe teacher knowledge about 
reading instruction. To provide support within an RTI model, teachers also need knowledge 
about how to deliver early, targeted, and systematic instruction in the foundational reading areas 
of vocabulary, phonics, phonemic awareness, fluency, and reading comprehension which have 
been shown to improve the literacy skills of struggling readers (National Reading Panel, 2000). 


However, converging findings from research surveying teachers have shown that teachers often 
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lack basic knowledge of how to teach foundational reading skills (e.g., Bos, Mather, Dickson, 
Podhajski, & Chard, 2001; Cunningham, Perry, Stanovich, & Stanovich, 2004; Moats & 
Foorman, 2003; Piasta, Connor, Fishman, & Morrison, 2009; Binks-Cantrell, Joshi, & 
Washburn, 2012). An early survey, the Informal Survey of Linguistic Knowledge developed by 
Moats (1994), has been foundational to many of the subsequent teacher knowledge surveys. This 
65-item survey focused mainly on teachers’ skills in spelling, terminology, phonology, and 
morphology. Results from her study, involving 89 participants, revealed that on average, novice 
and experienced teachers had poor knowledge about these key foundational reading and writing 
concepts (Moats, 1994). However, reliability data was not specified. 

Since then, several studies have been conducted showing converging results about the 
knowledge teachers have about teaching reading, with particular relevance to struggling readers 
and students with dyslexia (e.g., Bos et al., 2001; Moats & Foorman, 2003; Piasta et al., 2009; 
Carlisle, Kelcey, Rowan, & Phelps, 2011; Binks-Cantrell et al., 2012). For example, following 
publication of the National Reading Panel Report (2000), Bos et al. (2001) conducted a study to 
learn more about teachers’ knowledge about explicit teaching of reading. Bos and colleagues 
developed two measures: a 15-item survey, the Teacher Attitudes of Early Reading and Spelling 
(a ranging from 0.50 to 0.70), and the 20-item Teacher Knowledge assessment (0. = 0.60) with 
components modeled after the Moats (1994) survey. In the study, a total of 538 general and 
special education preservice and in-service teachers answered Likert-scaled questions about their 
perception of important reading skills and knowledge of foundational reading concepts. The 
researchers found that while special educators generally had higher knowledge than general 
educators, overall, all teachers incorrectly responded to nearly half of the questions regarding 


language structure. 


MEASURING TEACHER KNOWLEDGE ABOUT RTI 9 


A decade later, during Reading First, Carlisle et al. (2011) developed a survey that 
included some direct assessment of teachers’ knowledge to teach not only foundational linguistic 
skills but also reading comprehension, the Teacher Knowledge of Reading and Reading 
Practices. The survey was administered to over 1100 teachers who had received professional 
development during Reading First initiatives, and their data analysis explored the association 
between teachers’ level of knowledge and their students’ reading achievement. Carlisle and 
colleagues reported reliability results to be 0.76 for the coefficient alpha and 0.76 for IRT 
reliability. They found that first-grade students whose teachers were more knowledge- able had 
higher reading achievement. The researchers also found that the knowledge needed for teaching 
reading comprehension correlated with having knowledge about foundational linguistic skills. 

Across these studies, researchers have used surveys to assess teachers’ knowledge of the 
important elements of reading instruction (e.g., NRP, 2000). Findings consistently have shown 
that teachers lack necessary knowledge in foundational reading skills and the pedagogy to be 
able to teach reading effectively to prevent reading problems and to provide structured literacy 
interventions for students with reading disabilities, including dyslexia. Fortunately, several 
studies have demonstrated improved teacher knowledge has led to significantly stronger reading 
outcomes for their students (Bos, Mather, Narr, & Babur, 1999; Foorman & Moats, 2004; 
McCutchen, Abbott, et al., 2002; McCutchen, Harry, et al., 2002; Moats & Foorman, 2003; 
Spear-Swerling & Brucker, 2003, 2004). Bos et al. (1999) found that students improved reading 
skills when their teachers received specific professional development in early literacy more than 
students whose teachers did not receive the professional development. Similarly, McCutchen, 


Harry, et al. (2002) found in a randomized control trial that students whose teachers were 
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provided intensive training in foundational reading skills demonstrated higher reading gains at 
the end of the year than students of teachers in the control group. 
Factor Analytic Work for Teacher Surveys 

Further research is needed about the psychometric properties of surveys of RTI, 
particularly regarding factor structure as this would increase our understanding of the constructs 
related to teacher knowledge. Overall, researchers have modified existing survey instruments 
with the addition of questions or have developed original survey tools to measure teacher 
knowledge constructs of interest; however, there has been limited reporting for the psychometric 
features of the surveys or about the factors related to the constructs of interest. Some notable 
exceptions include two studies describing knowledge of reading instruction (Binks-Cantrell, 
Joshi, & Washburn, 2012; Phelps & Schilling, 2004) and one focusing on RTI knowledge 
(Barnes & Burchard, 2011) in which researchers detailed how each question type loaded onto 
specific factors identified as fundamental to the intended teacher knowledge construct. 

In one study, Binks-Cantrell, Joshi, & Washburn (2011) developed and tested the 
reliability of a 46-item multiple choice survey with 286 participants aimed at assessing teachers’ 
knowledge and skill in various language constructs important for students with dyslexia, 
including morphology, phonics, phonemic awareness, and phonological aware- ness. They found 
that the survey had high reliability in estimating where teachers lacked knowledge of language 
concepts (a = 0.90). Similarly, Phelps and Schilling (2004) developed the Content Knowledge 
for Teaching Reading and administered it to over 1500 teachers. Through factor analysis, the 
researchers determined that content knowledge for teaching reading included multiple 
dimensions (a ranged from 0.67 to 0.082). In a third study, Barnes and Burchard (2011) 


reported the results of their 58-item Multi- Tiered Instruction Self-Efficacy Scale in which they 
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asked 184 teachers and educational leaders about needs in implementing various components of 
RTI. The researchers determined the survey to be reliable (a = 0.98) in measuring educational 
professionals’ needs in broad concepts of RTI/MTSS. 

Although these three studies (Barnes & Burchard, 2011; Binks-Cantrell, Joshi, & 
Washburn, 2011; Phelps & Schilling, 2004) contribute to confidence that teacher knowledge 
surveys related to RTI and reading are reliable, additional work is needed to ensure good fit 
statistics. For example, these studies used Cronbach’s alpha; however, many researchers have 
argued that factor determinacy scores are a stronger statistic when measuring internal 
consistency (Lawley & Maxwell, 1963; Muliak, 1972; Raykov & Marcoulides, 2012; Sijtsma, 
2009). Using this knowledge, we planned to implement factor determinacy scores in our current 
study. In summary, teachers play an important role within RTI implementation, which affects 
reading outcomes for students with reading disabilities, including dyslexia. However, the 
existing research highlights that teachers need more support and professional development, 
particularly with regard to understanding how to use data to make decisions about appropriate 
evidence-based interventions and general knowledge of evidence-based practices in foundational 
early literacy instruction. 

Purpose and Research Questions 

The purpose of our exploratory study was to describe teachers’ knowledge about RTI 
with regard to their own ability to implement RTI and about their understanding about how RTI 
is implemented within their schools. One aim to inform this process was to measure teacher 
knowledge about RTI implementation reliably and validly. Another related aim was to determine 
the primary factors within this knowledge base that could be targeted for professional 


development. Both aims relate to the context of a larger study we are conducting, funded by the 
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Institute for Education Science, that will explore the relations among teachers’ knowledge of 
RTI, school interview data about RTI implementation, and students’ reading scores (blinded 
author grant funding no.). The larger project will not only describe these relations generally but 
will also focus on malleable factors to improve reading performance for struggling students. 

In the current study, we addressed three main research questions: (1) what was the factor 
analytic structure of the survey? (2) What was the internal consistency of the exploratory factor 


analysis of the survey? (3) What was the range of teacher scores on these factors? 


Method 

Participants and Settings 

Nine elementary schools were recruited during the second year of a 4-year larger study 
examining the relationships among RTI knowledge, practice, and reading outcomes. The purpose 
of the larger project is to identify effective RTI practices. These schools were located in four 
states near universities in the southwest, midwest, and northeast. The student population at the 
participating schools varied; at four schools, a majority of students received free and reduced 
price lunch, which is a proxy for socioeconomic status of their families. In these four schools, the 
percentage of students receiving free and reduced price lunch (FRPL) ranged from 72.4 to 94.1% 
percent, and in these schools, a majority of students were Hispanic. In another four schools, the 
percentage of students receiving FRPL was lower, ranging from 0.0 to 20%, and the majority of 
students were White. In the remaining school, the percentage of students receiving FRPL was 
37.8%, and the students were more diverse (with 36% African-American, 5.4% Hispanic, and the 


remainder were White). 
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We provided principals at each participating school the link to an online survey; 
principals provided the link to all teachers and staff at the school. Based upon data from school 
websites, we estimate that approximately, 283 teachers and staff would have received this link. 
Of those 283 possible respondents, 139 teachers and staff consented and responded anonymously 
for a response rate of 49.1%. One teacher did not finish the survey and was omitted from the data 
analysis. Teachers each reported years of experience (ranging from less than 1 year to more than 
35 with a median of 38% of teachers responding to have taught 5 years or less) and the grade 
level at which they were currently teaching (ranging from kindergarten to fifth grade). The 


number of respondents at each of the grade levels was evenly distributed across all grade levels. 


Survey and Procedures 


[Insert Table 1 here] 


We developed our survey of teachers’ knowledge about RTI implementation by first 
adapting and extending items from surveys developed by Roberts and his colleagues, which in 
turn, built upon existing survey items (Roberts, G., personal communication, June, 2014; (Bos et 
al., 2001; Edmonds, Roberts, & Vaughn, 2003; Oregon Reading First: Pilot surveys, 2003, Texas 
Reading First Baseline Surveys, 2004, Washington (RTI), 2011)). We also incorporated some 
items from the ECLS-K teacher surveys (c.f., Tourangeau, Lé, Nord, & Sorongon, 2009). We 
focused on items designed to capture teachers’ knowledge about RTI and their understanding 
about how RTI was implemented within their individual schools. The survey includes 52 items 
that teachers rate on a Likert scale (ranging from | = strongly disagree to 4 = strongly agree). For 
the convenience of our participants, the survey was administered using the online platform of 


Survey Monkey. See Table 1 for examples and Appendix Table 7 for all survey items. 
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Data Analytic Methods 


To address our research questions, we conducted an exploratory factor analysis (EFA) on 
the responses from the 138 teachers using Mplus (Muthen & Muthen, 1998). The goal of this 
analysis was to learn how many distinct factors emerged within our sample of participants and to 
evaluate the reliability and internal consistency of the items. According to Zwick and Velicer 
(1986), there are several procedures to determine the number of factors to retain. These include 
the eigenvalue > | rule (Kaiser, 1960), the scree test (Cattell, 1966), the minimum average partial 
correlation (Velicer, 1976), the Bartlett’s chi square test (Bartlett, 1950), and parallel analysis 
(Horn, 1965). Of these methods to determine the number of factors, the most widely used is the 
eigenvalue > | rule; however, this method may overestimate the number of retainable factors 
(Cattell & Jaspers, 1967; Linn, 1968; Zwick & Velicer, 1986). The scree test was more accurate, 
but still may overestimate factors (Thompson & Daniel, 1996). The least consistent method was 
Bartlett’s chi square test (Thompson & Daniel), whereas, parallel analysis appeared to be the best 
method for estimating the number of factors, while the minimum average partial correlation 


followed closely (Zwick & Velicer). 


We used parallel analysis to determine the number of factors in this survey. This method 
involved the following steps: (1) we observed eigenvalues were extracted from the original data 
set; (2) we simulated random data sets were generated and are parallel in structure to the original 
data set; (3) we extracted the eigenvalues from the parallel data sets; (4) 95th percentile 
distribution of eigenvalues were determined; (5) we observed eigenvalues and 95th percentile 
distribution of eigenvalues were compared; and (6) we retained a factor if the observed 
eigenvalue from the original data set was greater than the 95th percentile eigenvalue observed 


from the parallel data sets (Horn, 1965; Glorfeld, 1995). 
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After determining the number of factors contained within the survey, we performed an 
EFA to examine the factor loadings in order to place items to a corresponding factor. 
Researchers generally agree that a factor loading of 0.3 is the minimum used to determine 
whether or not an item belongs to a specific factor (e.g., Hair, Anderson, Tatham, & Black, 
1998), which we used as our threshold. From the EFA, we considered additional estimates in 
consideration of model fit, including y2, the comparative fit index (CFI) (Bentler, 1990), the 
standardized root mean square residual (SRMR), the root mean square error of approximation 
(RMSEA), and the Tucker—Lewis index (TLI) (Bollen, 1989; Jéreskog, 1993). According to Hu 
and Bentler (1999), an acceptable model fit was supported by a CFI value greater than or equal 
to 0.90, a SRMR value less than or equal to 0.08, a RMSEA value less than or equal to 0.05, and 


a TLI value greater than or equal to 0.95. 


Finally, instead of using the traditional Cronbach’s alpha, we calculated factor 
determinacy coefficients as a measure of internal consistency. The possible values range from 0 
to 1 with larger values indicating better measurement of the factor by the observed items. A 
factor determinacy coefficient of = 0.80 suggests strong correlation among items with their 
respective factor denoting high internal consistency (Lawley & Maxwell, 1963; Muliak, 1972). 
We chose this method because factor determinacy scores are considered to be more accurate than 
Cronbach’s alpha since Cronbach’s alpha is derived from the variances and covariances among 
the items rather than the parameters of the factor model (Raykov & Marcoulides, 2012; Sijtsma, 


2009). 


Results 


Research Question 1: what was the factor analytic structure of the RTI survey? 
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When conducting the factor analysis, we performed a parallel analysis to determine the 
number of factors contained within the RTI Survey of Teacher Knowledge. According to the 


parallel analysis, the strongest solution initially indicated four distinct factors as seen in Table 2. 


Following the guidance for factor analyses mentioned in the previous data analysis 
section, four factors emerged. However, when examining factor 4, the eigenvalues from the 
original correlation matrix were larger than the eigenvalues from the parallel analysis. Then, 
when looking at factor 5, both the average eigenvalues and 95th percentile eigenvalues from the 
parallel analysis were larger than the eigenvalues from the original correlation matrix. Due to 
these results, we conducted a four-factor exploratory analysis to examine the individual items. 


The rotated factor loadings are seen in Table 3. 


[Insert Table 3 here] 


When using the 0.3 threshold for minimum loading, we found 20 items for factor 1, 16 
items for factor 2, 4 items for factor 3, and 12 items for factor 4. Then, to label the constructs, we 
examined the individual items through a theoretical lens. We rank ordered the items in terms of 
the strength of their relation to the factors and named them accordingly. We noted that the four 
items within factor 3 appeared to relate to other factors but did not theoretically relate to each 
other. Specifically, items in questions 18, 19, and 37 appeared related to factor 1 and one item 
related to factor 4 (question 38). Question | remained without a true factor; however, it mostly 
loaded onto factor 1 (Teacher Knowledge about Tier 1 Implementation). Next, we named the 
factors Teacher Knowledge about Tier 1 Implementation (factor 1, 23 items), Teacher 
Knowledge about Leadership and School Systems, (factor 2, 16 items), and Teacher Knowledge 
about Data-Based Decision Making (factor 3, 13 items). We conducted a three-factor EFA. The 


rotated factor loadings are seen in Table 4. 
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[Insert Table 4 here] 


Next, we examined the model fit for both the 3 and 4 Factor EFAs found in Table 5 


[Insert Table 5 here] 


The 4 Factor EFA was somewhat better due to lower AIC, BIC, RMSEA, and SRMR and 
higher CFI and TLI, it also did not meet optimal standards as described in the data analytic 
methods. The three factor EFA was more consistent with our theoretical framework and it was 


also more parsimonious. 
Research Question 2: what was the internal consistency of the EFA? 


We used factor determinacy scores to analyze the internal consistency of the items within 
each factor, which can be seen in Table 6. Factor determinacy scores were above the cutoff, 


which indicated good reliability. 
[Insert Table 6 here] 
Research Question 3: what was the range of teachers’ scores on these factors? 


Possible responses for each item on the teacher survey had a range from 0 (the minimum, 
strongly disagree) to 4 (the maximum, strongly agree). We calculated composite scores for each 
of the three factors to see the level of teachers’ knowledge in those specific factors. For factor 1 
(Teacher Knowledge about Tier 1 Implementation), factor 2 (Teacher Knowledge about 
Leadership and School Systems, and factor 3 (Teacher Knowledge about Data-Based Decision 
Making), the composite scores were 3.08, 2.71, and 2.48 with standard deviations of 0.61, 0.86, 
and 0.91, respectively. The composite scores suggest that on average, teachers reported that they 


agreed that they are knowledgeable about implementing tier 1 of RTI and that are relatively 
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knowledgeable about Leadership and School Systems, with slightly more variability on 
Leadership and School Systems. On average, teachers reported being least knowledgeable about 
data-based decision making, and again, there was considerable variability in their answers. The 


full survey individual item means and standard deviations can be found in Appendix Table 8. 


Discussion 


Summary of Findings and Major Implications 


Our initial research question addressed the structure of the Survey of Teachers’ 
Knowledge about RTI Implementation. We addressed their teachers’ knowledge about how to 
implement RTI and also about their understanding of how RTI was implemented within their 
school. The three final factors from this factor analytic work were (1) Teacher Knowledge about 
Tier 1 Implementation, (2) Teacher Knowledge about Leadership and School Systems, and (3) 
Teacher Knowledge about Data-Based Decision Making. For example, items that loaded on the 
first factor, Teacher Knowledge related to Tier 1 implementation, included, for example, “This 
school has explicit goals for what we want students to achieve (learning outcomes) at each grade 
level” (q2) or “At this school, we know what the fidelity criteria are when instruction is 
observed” (q5). Items related to the second factor, Teacher Knowledge about Leadership and 
School Systems, included “Since implementing RTI, the number of referrals to special education 
has decreased” (q46) or “At this school, professional development related to RTI is coordinated” 
(q34). Items for the third factor, Teacher Knowledge about Data-Based Decision Making, 
included “Since implementing RTI, I have increased my expectations for my students’ academic 
performance (q23)” or “Since implementing RTI, at-risk students are identified early” (q29). 

These findings add to the current limited research about the factor structure of surveys to 


rate teachers’ knowledge generally, or specifically about RTI and reading (Binks-Cantrell, Joshi, 
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& Washburn, 2011; Phelps & Schilling, 2004; Barnes & Burchard, 2011). We modified items 
from two existing surveys, which Roberts and colleagues had shown were feasible for teachers to 
complete. To meet our research interests and further enhance the usability of the survey, we 
modified it by selecting the most relevant items, by adding additional items, and by creating an 
online version. An important implication is that it was feasible to use the online version, which 
can support collecting larger samples in future research related to training teachers to serve 
struggling readers, including students with dyslexia. An online version also increases the 
usability of the measure by school systems to assist them in making decisions about professional 
development. 

In addition, to answer our second research question, we examined the internal 
consistency of the items when placed into their subsequent factors discovered from the EFA. 
When analyzing the internal consistency of the items for each factor, we showed high internal 
consistency with determinacy scores above 0.9. Thus, we had no reason to remove any items at 
the present time. Practically, in terms of reliability and validity, it is important that our factors 
have high internal consistency. Strong internal consistency validates the strength of the specific 
factors, which will make understanding the results of the survey less complicated for teachers 
and other faculty working in the field who are using the survey. An implication is that the survey 
can reliably measure teacher knowledge, which is foundational for future research to identify 
areas for professional development to increase teachers’ knowledge about RTI, and ultimately to 
provide stronger intervention for students with dyslexia and other reading disabilities. 

Our third research question explored respondents’ knowledge on the three factors. We 
learned that there was considerable variability in teachers’ ratings of the knowledge, whereas the 


possible range was from 0 to 4, as seen in Appendix Table 8; the standard deviations on the 
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individual items ranged from 0.70 to 1.53. Findings indicate that teachers reported that they 
agreed that they were knowledgeable about tier 1 implementation (/ = 3.08; SD= 0.61) and that 
they were also relatively knowledgeable about leadership and school systems (M = 2.71; SD = 
.86), with slightly more variability. On average, they reported being least knowledgeable about 
data-based decision making (M = 2.48; SD = .91), and again, there was considerable variability 
in their answers. Notably, the composite scores showed less variability than did individual item 
scores. 

These findings add to and converge with findings from prior surveys indicating that 
teachers feel less prepared to make data-based instructional decisions than they report having 
broad knowledge about RTI (e.g., Barnes & Burchard, 2011; Spear-Swerling & Cheesman, 
2012; Vujnovic et al., 2014; Wilcox et al., 2013). This finding is problematic particularly for 
students who have or are at risk reading disabilities, and an implication is the need for further 
professional development. It is important to interpret these findings in light of the variability in 
RTI implementation and the lack of procedural guidance about RTI. 

Confirmation of the three individual factors and teachers’ reports of their own knowledge 
can guide future professional development to target specific weaknesses. We hope to inform 
the efforts of others who are working to increase teacher knowledge about RTI for their school. 
Because teachers have little knowledge of what to do with information about RTI to make 
instructional decisions (USDOE, 2009; Spear-Swerling & Cheesman, 2012), the individualized 
items and factors can provide a preliminary framework to help guide teachers and other 
instructional staff into making strong data-based decisions. It will also be easier for teachers to 
identify professional development opportunities to learn how to provide instruction to meet the 


needs of students with disabilities. 
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Limitations and Directions for Future Research 

The initial factor analysis from this pilot study shows promise; however, we do highlight 
several limitations. First, some of the specific fit indices from the factor analysis (CFI, TLI, etc.) 
were not optimal. There are a number of potential explanations for this, but we believe a large 
contributor to these less than optimal fit indices was our relatively small sample. In addition, 
some items reflect teachers’ own knowledge about RTI and others reflect their knowledge about 
RTI implementation within their schools. Small sample sizes have been shown to result in poor 
model fit; however, future research is underway to replicate this preliminary analysis with a 
larger sample size we have collected in a broader geographic area using a confirmatory factor 
analysis (CFA). We hypothesize that the factor loadings of the items will continue to show that 
the survey is a good fit for understanding teacher knowledge. In turn, with this larger sample 
size, model fit will improve. The CFA will also identify items that are redundant or not 
representative of a specific factor, which may allow us to shorten the survey to make it easier for 


teachers to complete in a timely manner. 


Another limitation is inherent to survey research. We are reporting what teachers self- 
report, and we have not measured their ability to use data, or their ability to accurately identify or 
intervention with students with reading disabilities, including dyslexia. Additionally, we were 
unable to report the teachers’ demographic information; the survey has been modified to 
include more demographic information about the teachers while still maintaining their 
anonymity. This information will include years of experience, specific dyslexia training, and 
training in special education, which will allow us to help schools identify sources of strength 
and needs for further training. Individual teachers could compare their own knowledge against a 


larger sample of teachers, which could help them identify professional development goals 
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to improve their knowledge to support students with or at risk for dyslexia and other reading 


disabilities. 


In addition, steps are already underway in a larger study to examine the correlations 
among these teacher knowledge about RTI factors, school level RTI implementation factors, and 
student reading growth. We are compiling a national database of students and their achievement 
scores, so it will be informative to link teacher knowledge about RTI and how students are 
performing. We hypothesize that there will be a strong positive correlation between teacher 
knowledge and students’ achievement scores. This examination will not only focus on students 
in tier 3, students receiving dyslexia services, and students receiving special education but will 


also examine the relations across a large and diverse sample. 
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Table 1 
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Factors and Examples 


Factors Example 1 Example 2 
Factor | Knowledge about Ex: This school has Ex: I am able to make 
Tier I Implementation explicit goals for instructional 
what we decisions based upon 
want students to assessment 
achieve (learning data. 
outcomes) at 
each grade level. 
Factor 2 Knowledge about Ex: Since Ex: Since 
School Systems implementing RTI, implementing RTI, 
there have been the 
fewer referrals to number of referrals to 
special education that special 
did not education has 
result in decreased. 
identification. 
Factor 3 Knowledge about Ex: Since Ex: Since 
Data-Based Decision implementing RTI, implementing RTI, I 
instructional have 
decisions are made increased my 
efficiently (i.e., expectations for my 


students do not 
have to wait to 
receive appropriate 
instruction). 


students' academic 
performance. 
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Table 2 

Eigenvalues From Parallel Analysis 

Factors 1 2 3 4 5 
Eigenvalues From Sample Correlation Matrix 18.44 492 256 2.35 1.84 
Average Eigenvalues From Parallel Analysis 246 2.31 2.19 2.09 2.00 
95" Percentile Eigenvalues From Parallel Analysis 2.63 243 2.30 2.17 2.08 
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Table 3 


Rotated Factor Loadings 4 Factor EFA 


Factors 
Question | 
Question 2 
Question 3 
Question 4 
Question 5 
Question 6 
Question 7 
Question 8 
Question 9 
Question 10 
Question 11 
Question 12 
Question 13 
Question 14 
Question 15 
Question 16 
Question 17 
Question 18 
Question 19 
Question 20 
Question 21 
Question 22 
Question 23 


Question 24 


0.174* 
0.943* 
0.956* 
-0.065 
0.008 
-0.064 
0.072 


-0.042 
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Question 25 
Question 26 
Question 27 
Question 28 
Question 29 
Question 30 
Question 31 
Question 32 
Question 33 
Question 34 
Question 35 
Question 36 
Question 37 
Question 38 
Question 39 
Question 40 
Question 41 
Question 42 
Question 43 
Question 44 
Question 45 
Question 46 
Question 47 
Question 48 
Question 49 
Question 50 


Question 51 


0.049 
-0.023 
0.034 
0.104 
-0.231* 
-0.064 
-0.192* 
0.368* 
0,335" 
0.109 
0252" 
-0.036 
0.14 
0.138 
0.225 
0.258* 
0.204 
0.191* 
0.08 
0.02 
-0.312* 
-0.195 
0.255 
0;335* 
0.152 
-0.222* 


-0.031 


0.792* 
0.563* 
0.763* 
0.724* 
0.807* 
0.678* 
0.607* 
0.246* 
0.207* 
O:310" 
0.458* 
0.403* 
-0.016 
-0.119 
-0.024 
0.083 
0.395% 
0.702* 
0.460* 
0537" 
0.295* 
0.23 
-0.058 
-0.014 
0.043 
0.469* 


0235" 


0.056 
0.03 
-0.059 
-0.006 
0.214* 
0.138 
0.048 
0.044 
0.113 
-0.006 
-0.023 
-0.022 
0.501* 
0222 
0.353* 
-0.019 
-0.159* 
-0.066 
-0.098 
-0.012 
-0.002 
-0.017 
0.075 
-0.029 
0.076 
0.185* 


0.102 


-0.03 
-0.034 
0.067 
0.063 
0.01 
0.04 
0.288* 
0.171 
0.245 
0.540* 
0.247* 
0.506* 
0.202* 
0.612* 
0.212 
0.530* 
0.359* 
0.017 
0.484* 
0.432* 
0.600* 
0:35)" 
0.519* 
0.559* 
0.558* 
0331" 


0.495* 
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Question 52 0.046 0.278* 


-0.097 


0.415* 
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Table 4 


Rotated Factor Loadings 4 Factor EFA 


Factors 1 2 3 
(Previously Factor 4) 

Question | 0.222* 0.063 0.168 
Question 2 0.658* 0.144 -0.062 
Question 3 0.608* 0.221* 0.035 
Question 4 0.411* 0.041 0.324* 
Question 5 0.409* 0.083 0.276* 
Question 6 0.575* -0.078 0.263* 
Question 7 0.612* 0.052 0.155 
Question 8 0.554* 0.115 0.063 
Question 9 0.532* 0.167 -0.01 
Question 10 0.650* 0.13 -0.107 
Question 11 0.502* -0.009 0.147 
Question 12 0.638* 0.13 -0.013 
Question 13 0.655* -0.024 0.096 
Question 14 0.531* 0.012 0.241* 
Question 15 0.499* 0.221* 0.108 
Question 16 0.392* 0.185 0.127 
Question 17 0.519* 0.405* -0.097 
Question 18 0.626* -0.035 -0.226 
Question 19 0.648* 0.002 -0.247 
Question 20 0.224* 0.623* 0.008 
Question 21 0.126 0.471* 0.219* 
Question 22 0.018 0.777* -0.075 


Question 23 0 0.885* -0.245* 
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Question 24 
Question 25 
Question 26 
Question 27 
Question 28 
Question 29 
Question 30 
Question 31 
Question 32 
Question 33 
Question 34 
Question 35 
Question 36 
Question 37 
Question 38 
Question 39 
Question 40 
Question 41 
Question 42 
Question 43 
Question 44 
Question 45 
Question 46 
Question 47 
Question 48 
Question 49 


Question 50 


-0.025 
0.053 
-0.049 
-0.024 
0.07 
-0.067 
0.031 
-0.139 
0:353* 
0.376* 
0.084 
O72" 
-0.059 
0.539* 
0351" 
0.511* 
0.251* 
0.033 
0.074 
-0.006 
0.011 
-0.247* 
-0.151 
0.346* 
0.339* 
0.232" 


-0.043 


0.751* 


0.806* 


0.545* 


O55" 


O72T* 


0.764* 


0.651* 


0.546* 


0.254* 


0.198* 


0.242* 


0.430* 


0.330* 


-0.012 


-0.152 


-0.005 


0.054 


0.357* 


0.699* 


0.408* 


0.488* 


0.21 


0.164 


-0.071 


-0.034 


-0.005 


0.413* 


0.117 
-0.027 
0.014 
0.106 
0.091 
-0.045 
0.016 
0.285* 
0.206* 
0.265* 
0.617* 
0.346* 
0.570* 
0.001 
0512* 
0.058 
0.556* 
0.499* 
0.113 
0.570* 
0.470* 
0.581* 
0.541* 
0.474* 
OS71% 
0.548* 


0.255" 
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Question 51 0.102 0.193 


Question 52 0.004 0:252* 


0.429* 


0.436* 


ay 
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Table 5 


Model Fit Indices For EFAs 
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3 Factor EFA 

AIC 1S272077 
BIC 19020.729 

’ pvalue 0.0000 
RMSEA 0.117 

CFI 0.613 

TLI 0.562 

SRMR 0.068 


4 Factor EFA 
18002.181 
18893.453 
0.0000 

0.111 

0.669 

0.610 

0.059 
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Table 6 


Factor Determinacy Scores for 3 Factor EFA 


Factors 1 2 


Question | .962 974 


954 


39 
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Appendix | 
Table 7 


Survey of Teachers’ Knowledge about RTI Implementation 


Survey Question 


ql This school is often one of the first to adopt a new program, practice, or 
educational model. 


q2 This school has explicit goals for what we want students to achieve (learning 
outcomes) at each grade level. 


q3 Goals and objectives for students' learning outcomes are being met at this 
school. 
q4 There is a system in place to ensure that all instruction (Tierl, 2, and 3) is 


being delivered with fidelity (1.e., faithfully, or as it was intended by the 
creator, developer, or publisher). 


qs At this school, we know what the fidelity criteria are (i.e., what is being 
looked for or at) when instruction is observed. 

q6 The administrators at this school spend time in class observing instruction. 

q7 Programs for improving student performance are coordinated at this school. 

q8 Everyone who delivers instruction (Tier 1, 2, and 3) to students in this school 
is knowledgeable. 

q9 All students receive Tier 1 core instruction in the general education 
classroom. 

ql0 Core instruction is designed to engage all students (e.g., differentiated or 


adapted to meet each student's needs) 


qll There is a systematic, formal, problem-solving process for intervening with 
struggling students at this school. 


ql2 Educators at this school are meeting the needs of struggling students. 

ql3 This school has a set of clear, predetermined grade-level benchmarks (i.e., cut 
scores, goal/targets, or percentiles) that are used to determine which students 
are at risk. 

ql4 This school has a set of clear, predetermined entry and exit criteria for all 


levels of instruction/intervention (core and interventions). 


ql5 At this school, we use data from each screening to ensure that Tier 1 core 
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ql6 


ql7 


ql8 
ql9 
q20 
q21 


q22 


q23 


q24 


q25 


q26 
q27 


q28 


q29 
q30 
q31 


q32 


q33 


q34 


instruction is meeting the needs of most of our students. 


At this school, administrators use assessment information to help make 
decisions about instruction. 


At this school, teachers use assessment information to help make decisions 
about instruction. 


I am able to interpret assessment data. 
I am able to make instruction decisions based upon assessment data. 
Since implementing RTI, instructional decisions are made in a timely manner. 


At this school, staff involved in RTI hold regular data meetings to discuss and 
analyze student progress data. 


Since implementing RTI, Iam more confident in my ability to meet the needs 
of my lowest students. 


Since implementing RTI, I have increased my expectations for my students' 
academic performance. 


Since implementing RTI, my school has increased expectations for my 
students' academic performance. 


Since implementing RTI, my school has increased our expectations for 
improving the reading skills of our students. 


Since implementing RTI, my workload has increased. 


Since implementing RTI, instructional decisions are made efficiently (i.e., 
students do not have to wait to receive appropriate instruction). 


Since implementing RTI, instructional decisions are made reliably (i.e., 
students receive instruction appropriate to their level of need). 


Since implementing RTI, at-risk students are identified early. 
Since implementing RTI, all at-risk students are identified. 


Since implementing RTI, fewer students who are not at risk have been 
identified as being at risk. 


Leaders in this school provide educators with resources they need to meet 
student achievement. 


Leaders at this school provide direction in matters of curriculum, teaching, 
and learning. 


At this school, professional development related to RTI is coordinated. 
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Educators at this school are prepared to implement RTI. 


Leaders in my school have been offered adequate opportunities to increase 
their ability to assist with RTI implementation. 


In general, I think I am capable of managing my instruction (e.g., 
differentiating instruction, pacing, etc.) 


There is a system (e.g., an "intervention team") in place for me to access help 
with managing my instruction. 


In general, I think I am capable of managing my students (e.g., behavior) 


There is a system (e.g., an "intervention team") in place for me to access help 
with classroom management. 


Educators at this school have an influence on RTI-related school policies and 
activities. 


Educators at this school work together to solve RTI-related challenges. 


Since implementing RTI, our school has a more widely shared common 
vision. 

Since implementing RTI, our students' learning outcomes have improved in 
all areas in which RTI was implemented. 


Since implementing RTI, the number of referrals to special education has 
decreased. 


Since implementing RTI, there have been fewer referrals to special education 
that did not result in identification. 


This school has a set of clear predetermined grade level benchmarks that are 
used to determine which students are struggling or at risk of failure in reading. 


This school has clear predetermined criteria for assigning students to a Tier of 
intervention. 


This school has clear predetermined criteria for exiting students from Tier 2 or 
Tier 3. 


This school uses RTI to evaluate whether students are eligible for special 
education services. 


This school has clear predetermined criteria for identifying students as having 
a learning disability in reading. 


This school places some students with the most intensive needs directly into 
Tier 3. 
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Appendix 2 
Table 8 
Descriptive Statistics for Survey of Teachers’ Knowledge about RTI Implementation 

Factor n M SD Min Max 

ql 1 135 2.63 1.24 0 4 
q2 1 134 3.30 0.92 0 4 
q3 1 135 3.03 0.94 0 4 
q4 1 135 2.70 1.17 0 4 
q5 1 135 2.99 0.97 0 4 
q6 1 133 2.94 1.15 0 4 
q7 1 135 3.07 0.94 0 4 
q8 1 135 2.99 1.10 0 4 
q9 1 135 321 1.02 0 4 
ql0 1 135 3.19 0.89 0 4 
qll 1 135 2.90 1.04 0 4 
ql2 1 135 2.96 0.92 0 4 
ql3 1 135 3.16 0.98 0 4 
ql4 1 135 2.93 1.02 0 4 
ql5 1 135 2.92 1.09 0 4 
ql6 1 135 3.08 0.92 0 4 
ql7 1 133 3.38 0.70 0 4 
ql8 1 135 3.39 0.86 0 4 
ql9 1 135 3.45 0.81 0 4 
q20 3 135 2.84 1.17 0 4 
q21 3 135 2.64 1.34 0 4 
q22 3 134 2.82 1.11 0 4 
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q50 3 135 2.60 
ql 2 135 2.53 
q52 2 135 2.35 


133 
1.40 
1.44 
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